Purpose

Previously, Paimon rejected changing the storage layout of an existing MAP
field between default and shared-shredding.

This restriction also prevented tables from gradually migrating their MAP
storage layout. After enabling shared shredding, a table can contain both old
default-layout MAP files and newly written shared-shredding files. Selected-key
pushdown must therefore be able to read both physical representations correctly.

This PR:

• Allows an existing MAP field to switch between default and
  shared-shredding while keeping the existing schema validation.
• Supports switching the layout in both directions:
  • default -> shared-shredding
  • shared-shredding -> default
  • switching back to shared-shredding
• Creates a selected-key read plan even when an old file does not contain
  shared-shredding metadata.
• Restores the physical MAP read type for selected-key reads of default-layout
  files.
• Projects the requested keys from a normal MapColumnVector, preserving the
  semantics of:
  • existing keys
  • missing keys
  • null MAP values
  • keys whose values are null
• Keeps selected-key pushdown enabled only when the current table layout is
  shared-shredding.
• Supports mixed-layout reads after MAP value-type evolution and column rename.
• Verifies that forced compaction rewrites old shared-shredding files into the
  current default layout.

Tests

Added and updated tests covering:

• MapSharedShreddingUtils.buildPhysicalReadType with:
  • no shared-shredding metadata
  • full MAP reads
  • selected-key reads from default-layout files
  • full and selected-key reads from shared-shredding files
  • field IDs, descriptions and nullability
• Selected-key materialization from normal MAP column vectors, including
  missing keys, null values and null MAP rows.
• Read-plan creation for selected-key reads of default-layout files.
• Schema changes in both directions, including changes combined with a column
  rename.
• ORC and Parquet reads across mixed default and shared-shredding files.
• MAP value-type evolution from INT to BIGINT.
• Full MAP and selected-key reads before and after layout changes.
• Forced compaction after switching from shared shredding to the default
  layout.
• Spark selected-key pushdown and result correctness across all layout
  generations.

The relevant common and core unit tests pass. The Spark E2E suite passes for
Spark 3.2, 3.3, 3.4, 3.5, 4.0 and 4.1 with both ORC and Parquet.